<!doctypehtml><html class="sidebar-visible no-js light"lang=en><head><meta charset=UTF-8><title>Line processing - Perl One-Liners Guide</title><meta content="text/html; charset=utf-8"http-equiv=Content-Type><meta content="Example based guide for text processing with Perl from the command line"name=description><meta content=width=device-width,initial-scale=1 name=viewport><meta content=#ffffff name=theme-color><meta content="Perl One-Liners Guide"property=og:title><meta content=website property=og:type><meta content="Example based guide for text processing with Perl from the command line"property=og:description><meta content=https://learnbyexample.github.io/learn_perl_oneliners/ property=og:url><meta content=https://raw.githubusercontent.com/learnbyexample/learn_perl_oneliners/main/images/perl_oneliners_ls.png property=og:image><meta content=1280 property=og:image:width><meta content=720 property=og:image:height><meta content=summary_large_image property=twitter:card><meta content=@learn_byexample property=twitter:site><link href="favicon.svg" rel=icon><link rel="shortcut icon"href="favicon.png"><link href="css/variables.css" rel=stylesheet><link href="css/general.css" rel=stylesheet><link href="css/chrome.css" rel=stylesheet><link href="FontAwesome/css/font-awesome.css" rel=stylesheet><link href="fonts/fonts.css" rel=stylesheet><link href="highlight.css" rel=stylesheet><link href="tomorrow-night.css" rel=stylesheet><link href="ayu-highlight.css" rel=stylesheet><link href="style.css" rel=stylesheet><body><script>var path_to_root = "";
            var default_theme = window.matchMedia("(prefers-color-scheme: dark)").matches ? "navy" : "light";</script><script>try {
                var theme = localStorage.getItem('mdbook-theme');
                var sidebar = localStorage.getItem('mdbook-sidebar');

                if (theme.startsWith('"') && theme.endsWith('"')) {
                    localStorage.setItem('mdbook-theme', theme.slice(1, theme.length - 1));
                }

                if (sidebar.startsWith('"') && sidebar.endsWith('"')) {
                    localStorage.setItem('mdbook-sidebar', sidebar.slice(1, sidebar.length - 1));
                }
            } catch (e) { }</script><script>var theme;
            try { theme = localStorage.getItem('mdbook-theme'); } catch(e) { }
            if (theme === null || theme === undefined) { theme = default_theme; }
            var html = document.querySelector('html');
            html.classList.remove('no-js')
            html.classList.remove('light')
            html.classList.add(theme);
            html.classList.add('js');</script><script>var html = document.querySelector('html');
            var sidebar = 'hidden';
            if (document.body.clientWidth >= 1080) {
                try { sidebar = localStorage.getItem('mdbook-sidebar'); } catch(e) { }
                sidebar = sidebar || 'visible';
            }
            html.classList.remove('sidebar-visible');
            html.classList.add("sidebar-" + sidebar);</script><nav aria-label="Table of contents"class=sidebar id=sidebar><div class=sidebar-scrollbox><ol class=chapter><li class="chapter-item expanded affix"><a href="cover.html">Cover</a><li class="chapter-item expanded affix"><a href="buy.html">Buy PDF/EPUB versions</a><li class="chapter-item expanded"><a href="preface.html"><strong aria-hidden=true>1.</strong> Preface</a><li class="chapter-item expanded"><a href="one-liner-introduction.html"><strong aria-hidden=true>2.</strong> One-liner introduction</a><li class="chapter-item expanded"><a class=active href="line-processing.html"><strong aria-hidden=true>3.</strong> Line processing</a><li class="chapter-item expanded"><a href="in-place-file-editing.html"><strong aria-hidden=true>4.</strong> In-place file editing</a><li class="chapter-item expanded"><a href="field-separators.html"><strong aria-hidden=true>5.</strong> Field separators</a><li class="chapter-item expanded"><a href="record-separators.html"><strong aria-hidden=true>6.</strong> Record separators</a><li class="chapter-item expanded"><a href="using-modules.html"><strong aria-hidden=true>7.</strong> Using modules</a><li class="chapter-item expanded"><a href="multiple-file-input.html"><strong aria-hidden=true>8.</strong> Multiple file input</a><li class="chapter-item expanded"><a href="processing-multiple-records.html"><strong aria-hidden=true>9.</strong> Processing multiple records</a><li class="chapter-item expanded"><a href="two-file-processing.html"><strong aria-hidden=true>10.</strong> Two file processing</a><li class="chapter-item expanded"><a href="dealing-with-duplicates.html"><strong aria-hidden=true>11.</strong> Dealing with duplicates</a><li class="chapter-item expanded"><a href="perl-rename-command.html"><strong aria-hidden=true>12.</strong> Perl rename command</a><li class="chapter-item expanded"><a href="Exercise_solutions.html"><strong aria-hidden=true>13.</strong> Exercise Solutions</a></li><br><hr><li class="chapter-item expanded"><i class="fa fa-github"id=git-repository-button></i><a href=https://github.com/learnbyexample/learn_perl_oneliners>   Source code</a><li class="chapter-item expanded"><i class="fa fa-home"id=home-button></i><a href="../index.html">   My Blog</a><li class="chapter-item expanded"><i class="fa fa-book"id=book-button></i><a href="../books.html">   My Books</a><li class="chapter-item expanded"><i class="fa fa-envelope"id=mail-button></i><a href=https://learnbyexample.gumroad.com/l/learnbyexample-weekly>   learnbyexample weekly</a><li class="chapter-item expanded"><i class="fa fa-twitter"id=twitter-button></i><a href=https://twitter.com/learn_byexample>   Twitter</a></ol></div><div class=sidebar-resize-handle id=sidebar-resize-handle></div></nav><div class=page-wrapper id=page-wrapper><div class=page><div id=menu-bar-hover-placeholder></div><div class="menu-bar sticky bordered"id=menu-bar><div class=left-buttons><button aria-label="Toggle Table of Contents"title="Toggle Table of Contents"aria-controls=sidebar class=icon-button id=sidebar-toggle type=button><i class="fa fa-bars"></i></button><button aria-label="Change theme"title="Change theme"aria-controls=theme-list aria-expanded=false aria-haspopup=true class=icon-button id=theme-toggle type=button><i class="fa fa-paint-brush"></i></button><ul aria-label=Themes class=theme-popup id=theme-list role=menu><li role=none><button class=theme id=light role=menuitem>Light (default)</button><li role=none><button class=theme id=rust role=menuitem>Rust</button><li role=none><button class=theme id=coal role=menuitem>Coal</button><li role=none><button class=theme id=navy role=menuitem>Navy</button><li role=none><button class=theme id=ayu role=menuitem>Ayu</button></ul><button aria-label="Toggle Searchbar"title="Search. (Shortkey: s)"aria-controls=searchbar aria-expanded=false aria-keyshortcuts=S class=icon-button id=search-toggle type=button><i class="fa fa-search"></i></button></div><h1 class=menu-title>Perl One-Liners Guide</h1><div class=right-buttons><a aria-label=Blog href="../index.html" title=Blog> <i class="fa fa-home"id=home-button></i> </a><a aria-label=Twitter href=https://twitter.com/learn_byexample title=Twitter> <i class="fa fa-twitter"id=twitter-button></i> </a><a aria-label="Git repository"title="Git repository"href=https://github.com/learnbyexample/learn_perl_oneliners> <i class="fa fa-github"id=git-repository-button></i> </a></div></div><div class=hidden id=search-wrapper><form class=searchbar-outer id=searchbar-outer><input placeholder="Search this book ..."aria-controls=searchresults-outer aria-describedby=searchresults-header id=searchbar name=searchbar type=search></form><div class="searchresults-outer hidden"id=searchresults-outer><div class=searchresults-header id=searchresults-header></div><ul id=searchresults></ul></div></div><script>document.getElementById('sidebar-toggle').setAttribute('aria-expanded', sidebar === 'visible');
                    document.getElementById('sidebar').setAttribute('aria-hidden', sidebar !== 'visible');
                    Array.from(document.querySelectorAll('#sidebar a')).forEach(function(link) {
                        link.setAttribute('tabIndex', sidebar === 'visible' ? 0 : -1);
                    });</script><div class=content id=content><main><div class=sidetoc><nav class=pagetoc></nav></div><h1 id=line-processing><a class=header href="line-processing.html#line-processing">Line processing</a></h1><p>Now that you are familiar with basic Perl CLI usage, this chapter will dive deeper into line processing examples. You'll learn various ways for matching lines based on regular expressions, fixed string matching, line numbers, etc. You'll also see how to group multiple statements and learn about the control flow keywords <code>next</code> and <code>exit</code>.<blockquote><p><img alt=info src="images/info.svg"> The <a href=https://github.com/learnbyexample/learn_perl_oneliners/tree/main/example_files>example_files</a> directory has all the files used in the examples.</blockquote><h2 id=regexp-based-filtering><a class=header href="line-processing.html#regexp-based-filtering">Regexp based filtering</a></h2><p>As mentioned before:<ul><li><code>/REGEXP/FLAGS</code> is a shortcut for <code>$_ =~ m/REGEXP/FLAGS</code><li><code>!/REGEXP/FLAGS</code> is a shortcut for <code>$_ !~ m/REGEXP/FLAGS</code></ul><p>Here are some examples:<pre><code class=language-bash>$ cat table.txt
brown bread mat hair 42
blue cake mug shirt -7
yellow banana window shoes 3.14

$ perl -ne 'print if /ow\b/' table.txt
yellow banana window shoes 3.14

$ perl -ne 'print if !/[ksy]/' table.txt
brown bread mat hair 42
</code></pre><p>If required, you can also use different delimiters. Quoting from <a href=https://perldoc.perl.org/perlop#m/PATTERN/msixpodualngc>perldoc: match</a>:<blockquote><p>If <code>/</code> is the delimiter then the initial <code>m</code> is optional. With the <code>m</code> you can use any pair of non-whitespace (ASCII) characters as delimiters. This is particularly useful for matching path names that contain <code>/</code>, to avoid LTS (<strong>leaning toothpick syndrome</strong>). If <code>?</code> is the delimiter, then a match-only-once rule applies, described in <code>m?PATTERN?</code> below. If <code>'</code> (single quote) is the delimiter, no variable interpolation is performed on the <em>PATTERN</em>. When using a delimiter character valid in an identifier, whitespace is required after the <code>m</code>. <em>PATTERN</em> may contain variables, which will be interpolated every time the pattern search is evaluated, except for when the delimiter is a single quote.</blockquote><pre><code class=language-bash>$ cat paths.txt
/home/joe/report.log
/home/ram/power.log
/home/rambo/errors.log

# leaning toothpick syndrome
$ perl -ne 'print if /\/home\/ram\//' paths.txt
/home/ram/power.log

# using a different delimiter makes it more readable here
$ perl -ne 'print if m{/home/ram/}' paths.txt
/home/ram/power.log

$ perl -ne 'print if !m#/home/ram/#' paths.txt
/home/joe/report.log
/home/rambo/errors.log
</code></pre><h2 id=extracting-matched-portions><a class=header href="line-processing.html#extracting-matched-portions">Extracting matched portions</a></h2><p>You can use regexp related special variables to extract only the matching portions. Consider this input file:<pre><code class=language-bash>$ cat ip.txt
it is a warm and cozy day
listen to what I say
go play in the park
come back before the sky turns dark

There are so many delights to cherish
Apple, Banana and Cherry
Bread, Butter and Jelly
Try them all before you perish
</code></pre><p>Here are some examples of extracting only the matched portions.<pre><code class=language-bash># note that this will print only the first match for each input line
$ perl -nE 'say $& if /\b[a-z]\w*[ty]\b/' ip.txt
it
what
play
sky
many

$ perl -nE 'say join "::", @{^CAPTURE} if /(\b[bdp]\w+).*((?1))/i' ip.txt
play::park
back::dark
Bread::Butter
before::perish
</code></pre><p>Special variables to work with capture groups aren't always needed. For example, when every line has a match.<pre><code class=language-bash>$ perl -nE 'say /^(\w+ ).*?(\d+)$/' table.txt
brown 42
blue 7
yellow 14

# with a custom separator
$ perl -nE 'say join ":", /^(\w+).*?(\d+)$/' table.txt
brown:42
blue:7
yellow:14
</code></pre><h2 id=transliteration><a class=header href="line-processing.html#transliteration">Transliteration</a></h2><p>The transliteration operator <code>tr</code> (or <code>y</code>) helps you perform transformations character-wise. See <a href=https://perldoc.perl.org/perlop#tr/SEARCHLIST/REPLACEMENTLIST/cdsr>perldoc: tr</a> for documentation.<pre><code class=language-bash># rot13
$ echo 'Uryyb Jbeyq' | perl -pe 'tr/a-zA-Z/n-za-mN-ZA-M/'
Hello World

# 'c' option complements the specified characters
$ echo 'apple:123:banana' | perl -pe 'tr/0-9\n/-/c'
------123-------

# 'd' option deletes the characters
$ echo 'apple:123:banana' | perl -pe 'tr/0-9\n//cd'
123

# 's' option squeezes repeated characters
$ echo 'APPLE gobbledygook' | perl -pe 'tr|A-Za-z||s'
APLE gobledygok
# transliteration as well as squeeze
$ echo 'APPLE gobbledygook' | perl -pe 'tr|A-Z|a-z|s'
aple gobbledygook
</code></pre><p>Similar to the <code>s</code> operator, <code>tr</code> returns the number of changes made. Use the <code>r</code> option to prevent in-place modification and return the transliterated string instead.<pre><code class=language-bash># match lines containing 'b' 2 times
$ perl -ne 'print if tr/b// == 2' table.txt
brown bread mat hair 42

$ s='orange apple appleseed'
$ echo "$s" | perl -pe 's#\bapple\b(*SKIP)(*F)|\w+#$&=~tr/a-z/A-Z/r#ge'
ORANGE apple APPLESEED
</code></pre><p>See also:<ul><li><a href=https://stackoverflow.com/q/45571828/4082052>stackoverflow: reverse complement DNA sequence for a specific field</a><li><a href=https://unix.stackexchange.com/q/396584/109046>unix.stackexchange: count the number of characters except specific characters</a><li><a href=https://unix.stackexchange.com/q/428085/109046>unix.stackexchange: scoring DNA data</a></ul><h2 id=conditional-substitution><a class=header href="line-processing.html#conditional-substitution">Conditional substitution</a></h2><p>These examples combine line filtering and substitution in different ways. As noted before, the <code>s</code> operator modifies the input string and the return value can be used to know how many substitutions were made. Use the <code>r</code> flag to prevent in-place modification and get the string output after substitution.<pre><code class=language-bash># change commas to hyphens if the input line does NOT contain '2'
# prints all input lines even if the substitution fails
$ printf '1,2,3,4\na,b,c,d\n' | perl -pe 's/,/-/g if !/2/'
1,2,3,4
a-b-c-d

# perform substitution only for the filtered lines
# prints filtered input lines, even if the substitution fails
$ perl -ne 'print s/ark/[$&]/rg if /the/' ip.txt
go play in the p[ark]
come back before the sky turns d[ark]
Try them all before you perish

# print only if the substitution succeeds
$ perl -ne 'print if s/\bw\w*t\b/{$&}/g' ip.txt
listen to {what} I say
</code></pre><h2 id=multiple-conditions><a class=header href="line-processing.html#multiple-conditions">Multiple conditions</a></h2><p>It is good to remember that Perl is a programming language. You can make use of control structures and combine multiple conditions using logical operators. You don't have to create a single complex regexp.<pre><code class=language-bash>$ perl -ne 'print if /ark/ && !/sky/' ip.txt
go play in the park

$ perl -ane 'print if /\bthe\b/ || $#F == 5' ip.txt
go play in the park
come back before the sky turns dark
Try them all before you perish

$ perl -ne 'print if /s/ xor /m/' table.txt
brown bread mat hair 42
yellow banana window shoes 3.14
</code></pre><h2 id=next><a class=header href="line-processing.html#next">next</a></h2><p>When the <code>next</code> statement is executed, rest of the code will be skipped and the next input line will be fetched for processing. It doesn't affect the <code>BEGIN</code> and <code>END</code> blocks as they are outside the file content loop.<pre><code class=language-bash>$ perl -nE 'if(/\bpar/){print "%% $_"; next} say /s/ ? "X" : "Y"' anchors.txt
%% sub par
X
Y
X
%% cart part tart mart
</code></pre><p><strong>Note</strong> that <code>{}</code> is used in the above example to group multiple statements to be executed for a single <code>if</code> condition. You'll see many more examples with <code>next</code> in the coming chapters.<h2 id=exit><a class=header href="line-processing.html#exit">exit</a></h2><p>The <code>exit</code> function is useful to avoid processing unnecessary input content when a termination condition is reached. See <a href=https://perldoc.perl.org/functions/exit>perldoc: exit</a> for documentation.<pre><code class=language-bash># quits after an input line containing 'say' is found
$ perl -ne 'print; exit if /say/' ip.txt
it is a warm and cozy day
listen to what I say

# the matching line won't be printed in this case
$ perl -pe 'exit if /say/' ip.txt
it is a warm and cozy day
</code></pre><p>Use <code>tac</code> to get all lines starting from the last occurrence of the search string in the entire file.<pre><code class=language-bash>$ tac ip.txt | perl -ne 'print; exit if /an/' | tac
Bread, Butter and Jelly
Try them all before you perish
</code></pre><p>You can optionally provide a status code as an argument to the <code>exit</code> function.<pre><code class=language-bash>$ printf 'sea\neat\ndrop\n' | perl -ne 'print; exit(2) if /at/'
sea
eat
$ echo $?
2
</code></pre><p>Any code in the <code>END</code> block will still be executed before exiting. This doesn't apply if <code>exit</code> was called from the <code>BEGIN</code> block.<pre><code class=language-bash>$ perl -pE 'exit if /cake/' table.txt
brown bread mat hair 42

$ perl -pE 'exit if /cake/; END{say "bye"}' table.txt
brown bread mat hair 42
bye

$ perl -pE 'BEGIN{say "hi"; exit; say "hello"} END{say "bye"}' table.txt
hi
</code></pre><blockquote><p><img alt=warning src="images/warning.svg"> Be careful if you want to use <code>exit</code> with multiple input files, as Perl will stop even if there are other files remaining to be processed.</blockquote><h2 id=line-number-based-processing><a class=header href="line-processing.html#line-number-based-processing">Line number based processing</a></h2><p>Line numbers can also be specified as a matching criteria by using the <code>$.</code> special variable.<pre><code class=language-bash># print only the third line
$ perl -ne 'print if $. == 3' ip.txt
go play in the park

# print the second and sixth lines
$ perl -ne 'print if $. == 2 || $. == 6' ip.txt
listen to what I say
There are so many delights to cherish

# transliterate only the second line
$ printf 'gates\nnot\nused\n' | perl -pe 'tr/a-z/*/ if $. == 2'
gates
***
used

# print from a particular line number to the end of the input
$ seq 14 25 | perl -ne 'print if $. >= 10'
23
24
25
</code></pre><p>Use the <code>eof</code> function to check for the end of the file condition. See <a href=https://perldoc.perl.org/perlfunc#eof>perldoc: eof</a> for documentation.<pre><code class=language-bash># same as: tail -n1 ip.txt
$ perl -ne 'print if eof' ip.txt
Try them all before you perish

$ perl -ne 'print "$.:$_" if eof' ip.txt
9:Try them all before you perish

# multiple file example
# same as: tail -q -n1 ip.txt table.txt
$ perl -ne 'print if eof' ip.txt table.txt
Try them all before you perish
yellow banana window shoes 3.14
</code></pre><p>For large input files, you can use <code>exit</code> to avoid processing unnecessary input lines.<pre><code class=language-bash>$ seq 3542 4623452 | perl -ne 'if($. == 2452){print; exit}'
5993
$ seq 3542 4623452 | perl -ne 'print if $. == 250; if($. == 2452){print; exit}'
3791
5993

# here is a sample time comparison
$ time seq 3542 4623452 | perl -ne 'if($. == 2452){print; exit}' > f1
real    0m0.005s
$ time seq 3542 4623452 | perl -ne 'print if $. == 2452' > f2
real    0m0.496s
$ rm f1 f2
</code></pre><h2 id=range-operator><a class=header href="line-processing.html#range-operator">Range operator</a></h2><p>You can use the range operator to select between a pair of matching conditions like line numbers and regexp. See <a href=https://perldoc.perl.org/perlop#Range-Operators>perldoc: range</a> for documentation.<pre><code class=language-bash># the range is automatically compared against $. in this context
# same as: perl -ne 'print if 3 <= $. <= 5'
$ seq 14 25 | perl -ne 'print if 3..5'
16
17
18

# the range is automatically compared against $_ in this context
# note that all the matching ranges are printed
$ perl -ne 'print if /to/ .. /pl/' ip.txt
listen to what I say
go play in the park
There are so many delights to cherish
Apple, Banana and Cherry
</code></pre><blockquote><p><img alt=info src="images/info.svg"> See the <a href="processing-multiple-records.html#records-bounded-by-distinct-markers">Records bounded by distinct markers</a> section for an alternate solution.</blockquote><p>Line numbers and regexp filtering can be mixed.<pre><code class=language-bash>$ perl -ne 'print if 6 .. /utter/' ip.txt
There are so many delights to cherish
Apple, Banana and Cherry
Bread, Butter and Jelly

# same logic as: perl -pe 'exit if /\bba/'
# inefficient, but this will work for multiple file inputs
$ perl -ne 'print if !(/\bba/ .. eof)' ip.txt table.txt
it is a warm and cozy day
listen to what I say
go play in the park
brown bread mat hair 42
blue cake mug shirt -7
</code></pre><p>Both conditions can match the same line too! Use <code>...</code> if you don't want the second condition to be matched against the starting line. Also, if the second condition doesn't match, lines starting from the first condition to the last line of the input will be matched.<pre><code class=language-bash># 'and' matches the 7th line
$ perl -ne 'print if 7 .. /and/' ip.txt
Apple, Banana and Cherry

# 'and' will be tested against 8th line onwards
$ perl -ne 'print if 7 ... /and/' ip.txt
Apple, Banana and Cherry
Bread, Butter and Jelly

# there's a line containing 'Banana' but the matching pair isn't found
# so, all lines till the end of the input is printed
$ perl -ne 'print if /Banana/ .. /XYZ/' ip.txt
Apple, Banana and Cherry
Bread, Butter and Jelly
Try them all before you perish
</code></pre><h2 id=working-with-fixed-strings><a class=header href="line-processing.html#working-with-fixed-strings">Working with fixed strings</a></h2><p>You can surround a regexp pattern with <code>\Q</code> and <code>\E</code> to match it as a fixed string, similar to the <code>grep -F</code> option. <code>\E</code> can be left out if there's no further pattern to be specified. Variables are still interpolated, so if your fixed string contains <code>$</code> or <code>@</code> forming possible variables, you'll run into issues. For such cases, one workaround is to pass the search string as an environment value and then apply <code>\Q</code> to that variable. See <a href=https://perldoc.perl.org/functions/quotemeta>perldoc: quotemeta</a> for documentation.<pre><code class=language-bash># no match, since [] are character class metacharacters
$ printf 'int a[5]\nfig\n1+4=5\n' | perl -ne 'print if /a[5]/'

$ perl -E 'say "\Qa[5]"'
a\[5\]
$ printf 'int a[5]\nfig\n1+4=5\n' | perl -ne 'print if /\Qa[5]/'
int a[5]
$ printf 'int a[5]\nfig\n1+4=5\n' | perl -pe 's/\Qa[5]/b[12]/'
int b[12]
fig
1+4=5

# $y and $z will be treated as uninitialized variables here
$ echo '$x = $y + $z' | perl -pe 's/\Q$y + $z/100/'
$x = $y100$z
$ echo '$x = $y + $z' | fs='$y + $z' perl -pe 's/\Q$ENV{fs}/100/'
$x = 100
# ENV is preferred since \\ is special in single quoted strings
$ perl -E '$x = q(x\y\\0z); say $x'
x\y\0z
$ x='x\y\\0z' perl -E 'say $ENV{x}'
x\y\\0z
</code></pre><p>If you just want to filter a line based on fixed strings, you can also use the <code>index</code> function. This returns the matching position (which starts with <code>0</code>) and <code>-1</code> if the given string wasn't found. See <a href=https://perldoc.perl.org/functions/index>perldoc: index</a> for documentation.<pre><code class=language-bash>$ printf 'int a[5]\nfig\n1+4=5\n' | perl -ne 'print if index($_, "a[5]") != -1'
int a[5]
</code></pre><p>The above <code>index</code> example uses double quotes for the string argument, which allows escape sequences like <code>\t</code>, <code>\n</code>, etc and interpolation. This isn't the case with single quoted string values. Using single quotes within the script from command line requires messing with shell metacharacters. So, use the <code>q</code> operator instead or pass the fixed string to be matched as an environment variable.<pre><code class=language-bash># double quotes allow escape sequences and interpolation
$ perl -E '$x=5; say "value of x:\t$x"'
value of x:     5

# use the 'q' operator as an alternate for single quoted strings
$ s='$a = 2 * ($b + $c)'
$ echo "$s" | perl -ne 'print if index($_, q/($b + $c)/) != -1'
$a = 2 * ($b + $c)

# or pass the string as an environment variable
$ echo "$s" | fs='($b + $c)' perl -ne 'print if index($_, $ENV{fs}) != -1'
$a = 2 * ($b + $c)
</code></pre><p>You can use the return value of the <code>index</code> function to restrict the matching to the start or end of the input line. The line content in the <code>$_</code> variable contains the <code>\n</code> line ending character as well. You can remove the line separator using the <code>chomp</code> function or the <code>-l</code> command line option (which will be discussed in detail in the <a href="record-separators.html#record-separators">Record separators</a> chapter). For now, it is enough to know that <code>-l</code> will remove the line separator and add it back when <code>print</code> is used.<pre><code class=language-bash>$ cat eqns.txt
a=b,a-b=c,c*d
a+b,pi=3.14,5e12
i*(t+9-g)/8,4-a+b

# start of the line
$ s='a+b' perl -ne 'print if index($_, $ENV{s})==0' eqns.txt
a+b,pi=3.14,5e12

# end of the line
# same as: s='a+b' perl -ne 'print if /\Q$ENV{s}\E$/' eqns.txt
# length function returns the number of characters, by default acts on $_
# -l option is needed here to remove \n from $_
$ s='a+b' perl -lne '$pos = length() - length($ENV{s});
                     print if index($_, $ENV{s}) == $pos' eqns.txt
i*(t+9-g)/8,4-a+b
</code></pre><p>Here are some more examples using the return value of the <code>index</code> function.<pre><code class=language-bash># since 'index' returns '-1' if there's no match,
# you need to add >=0 check as well for < or <= comparison
$ perl -ne '$i = index($_, "="); print if 0 <= $i <= 5' eqns.txt
a=b,a-b=c,c*d

# > or >= comparison is easy to specify
# if you pass the third argument to 'index', you'll still have to check != -1
$ s='a+b' perl -ne 'print if index($_, $ENV{s})>=1' eqns.txt
i*(t+9-g)/8,4-a+b
</code></pre><p>If you need to match the entire input line or a particular field, you can use the string comparison operators.<pre><code class=language-bash>$ printf 'a.b\na+b\n' | perl -lne 'print if /^a.b$/'
a.b
a+b
$ printf 'a.b\na+b\n' | perl -lne 'print if $_ eq q/a.b/'
a.b
$ printf '1 a.b\n2 a+b\n' | perl -lane 'print if $F[1] ne q/a.b/'
2 a+b
</code></pre><p>To provide a fixed string in the replacement section, environment variables come in handy again. Or, use the <code>q</code> operator for directly providing the value, but you may have to workaround the delimiters being used and the presence of <code>\\</code> characters.<pre><code class=language-bash># characters like $ and @ are special in the replacement section
$ echo 'x+y' | perl -pe 's/\Qx+y/$x+@y/'
+

# provide replacement string as an environment variable
$ echo 'x+y' | r='$x+@y' perl -pe 's/\Qx+y/$ENV{r}/'
$x+@y

# or, use the 'e' flag to provide a single quoted value as Perl code
$ echo 'x+y' | perl -pe 's/\Qx+y/q($x+@y)/e'
$x+@y

# need to workaround delimiters and \\ for the 'q' operator based solution
$ echo 'x+y' | perl -pe 's/\Qx+y/q($x\/@y)/e'
$x/@y
$ echo 'x+y' | perl -pe 's|\Qx+y|q($x/@y)|e'
$x/@y
$ echo 'x+y' | perl -pe 's|\Qx+y|q($x/@y\\\z)|e'
$x/@y\\z
</code></pre><h2 id=summary><a class=header href="line-processing.html#summary">Summary</a></h2><p>This chapter showed various examples of processing only the lines of interest instead of the entire input file. Filtering can be specified using a regexp, fixed string, line number or a combination of them. The <code>next</code> and <code>exit</code> statements are useful to change the flow of code.<h2 id=exercises><a class=header href="line-processing.html#exercises">Exercises</a></h2><blockquote><p><img alt=info src="images/info.svg"> The <a href=https://github.com/learnbyexample/learn_perl_oneliners/tree/main/exercises>exercises</a> directory has all the files used in this section.</blockquote><p><strong>1)</strong> For the given input, display except the third line.<pre><code class=language-bash>$ seq 34 37 | ##### add your solution here
34
35
37
</code></pre><p><strong>2)</strong> Display only the fourth, fifth, sixth and seventh lines for the given input.<pre><code class=language-bash>$ seq 65 78 | ##### add your solution here
68
69
70
71
</code></pre><p><strong>3)</strong> For the input file <code>ip.txt</code>, replace all occurrences of <code>are</code> with <code>are not</code> and <code>is</code> with <code>is not</code> only from line number <strong>4</strong> till the end of file. Also, only the lines that were changed should be displayed in the output.<pre><code class=language-bash>$ cat ip.txt
Hello World
How are you
This game is good
Today is sunny
12345
You are funny

##### add your solution here
Today is not sunny
You are not funny
</code></pre><p><strong>4)</strong> For the given stdin, display only the first three lines. Avoid processing lines that are not relevant.<pre><code class=language-bash>$ seq 14 25 | ##### add your solution here
14
15
16
</code></pre><p><strong>5)</strong> For the input file <code>ip.txt</code>, display all lines from the start of the file till the first occurrence of <code>game</code>.<pre><code class=language-bash>##### add your solution here
Hello World
How are you
This game is good
</code></pre><p><strong>6)</strong> For the input file <code>ip.txt</code>, display all lines that contain <code>is</code> but not <code>good</code>.<pre><code class=language-bash>##### add your solution here
Today is sunny
</code></pre><p><strong>7)</strong> For the input file <code>ip.txt</code>, extract the word before the whole word <code>is</code> as well as the word after it. If such a match is found, display the two words around <code>is</code> in reversed order. For example, <code>hi;1 is--234 bye</code> should be converted to <code>234:1</code>. Assume that the whole word <code>is</code> will not be present more than once in a single line.<pre><code class=language-bash>##### add your solution here
good:game
sunny:Today
</code></pre><p><strong>8)</strong> For the input file <code>hex.txt</code>, replace all occurrences of <code>0xA0</code> with <code>0x50</code> and <code>0xFF</code> with <code>0x7F</code>.<pre><code class=language-bash>$ cat hex.txt
start: 0xA0, func1: 0xA0
end: 0xFF, func2: 0xB0
restart: 0xA010, func3: 0x7F

##### add your solution here
start: 0x50, func1: 0x50
end: 0x7F, func2: 0xB0
restart: 0x5010, func3: 0x7F
</code></pre><p><strong>9)</strong> Find the starting index of the first occurrence of <code>is</code> or <code>the</code> or <code>was</code> or <code>to</code> for each input line of the file <code>idx.txt</code>. Assume that every input line will match at least one of these terms.<pre><code class=language-bash>$ cat idx.txt
match after the last newline character
and then you want to test
this is good bye then
you were there to see?

##### add your solution here
12
4
2
9
</code></pre><p><strong>10)</strong> Display all lines containing <code>[4]*</code> for the given stdin data.<pre><code class=language-bash>$ printf '2.3/[4]*6\n2[4]5\n5.3-[4]*9\n' | ##### add your solution here
2.3/[4]*6
5.3-[4]*9
</code></pre><p><strong>11)</strong> For the given input string, replace all lowercase alphabets to <code>x</code> only for words starting with <code>m</code>.<pre><code class=language-bash>$ s='ma2T3a a2p kite e2e3m meet'
$ echo "$s" | ##### add your solution here
xx2T3x a2p kite e2e3m xxxx
</code></pre><p><strong>12)</strong> For the input file <code>ip.txt</code>, delete all characters other than lowercase vowels and the newline character. Perform this transformation only between a line containing <code>you</code> up to line number <code>4</code> (inclusive).<pre><code class=language-bash>##### add your solution here
Hello World
oaeou
iaeioo
oaiu
12345
You are funny
</code></pre><p><strong>13)</strong> For the input file <code>sample.txt</code>, display from the start of the file till the first occurrence of <code>are</code>, excluding the matching line.<pre><code class=language-bash>$ cat sample.txt
Hello World

Good day
How are you

Just do-it
Believe it

Today is sunny
Not a bit funny
No doubt you like it too

Much ado about nothing
He he he

##### add your solution here
Hello World

Good day
</code></pre><p><strong>14)</strong> For the input file <code>sample.txt</code>, display from the last occurrence of <code>do</code> till the end of the file.<pre><code class=language-bash>##### add your solution here
Much ado about nothing
He he he
</code></pre><p><strong>15)</strong> For the input file <code>sample.txt</code>, display from the 9th line till a line containing <code>you</code>.<pre><code class=language-bash>##### add your solution here
Today is sunny
Not a bit funny
No doubt you like it too
</code></pre><p><strong>16)</strong> Display only the odd numbered lines from <code>ip.txt</code>.<pre><code class=language-bash>##### add your solution here
Hello World
This game is good
12345
</code></pre><p><strong>17)</strong> For the <code>table.txt</code> file, print only the line number for lines containing <code>air</code> or <code>win</code>.<pre><code class=language-bash>$ cat table.txt
brown bread mat hair 42
blue cake mug shirt -7
yellow banana window shoes 3.14

##### add your solution here
1
3
</code></pre><p><strong>18)</strong> For the input file <code>table.txt</code>, calculate the sum of numbers in the last column, excluding the second line.<pre><code class=language-bash>##### add your solution here
45.14
</code></pre><p><strong>19)</strong> Print the second and fourth line for every block of five lines.<pre><code class=language-bash>$ seq 15 | ##### add your solution here
2
4
7
9
12
14
</code></pre><p><strong>20)</strong> For the input file <code>ip.txt</code>, display all lines containing <code>e</code> or <code>u</code> but not both.<pre><code class=language-bash>##### add your solution here
Hello World
This game is good
Today is sunny
</code></pre></main><nav aria-label="Page navigation"class=nav-wrapper><a aria-label="Previous chapter"class="mobile-nav-chapters previous"title="Previous chapter"aria-keyshortcuts=Left href="one-liner-introduction.html" rel=prev> <i class="fa fa-angle-left"></i> </a><a aria-label="Next chapter"class="mobile-nav-chapters next"title="Next chapter"aria-keyshortcuts=Right href="in-place-file-editing.html" rel=next> <i class="fa fa-angle-right"></i> </a><div style="clear: both"></div></nav></div></div><nav aria-label="Page navigation"class=nav-wide-wrapper><a aria-label="Previous chapter"class="nav-chapters previous"title="Previous chapter"aria-keyshortcuts=Left href="one-liner-introduction.html" rel=prev> <i class="fa fa-angle-left"></i> </a><a aria-label="Next chapter"class="nav-chapters next"title="Next chapter"aria-keyshortcuts=Right href="in-place-file-editing.html" rel=next> <i class="fa fa-angle-right"></i> </a></nav></div><script>window.playground_copyable = true;</script><script charset=utf-8 src="elasticlunr.min.js"></script><script charset=utf-8 src="mark.min.js"></script><script charset=utf-8 src="searcher.js"></script><script charset=utf-8 src="clipboard.min.js"></script><script charset=utf-8 src="highlight.js"></script><script charset=utf-8 src="book.js"></script><script src="sidebar.js"></script>